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Abstract 

Extremum seeking feedback is a powerful method to steer a dynamical system to an extremum 
of a partially or completely unknown map. It often requires advanced system-theoretic tools to 
understand the qualitative behavior of extremum seeking systems. In this paper, a novel interpre- 
tation of extremum seeking is introduced. We show that the trajectories of an extremum seeking 
system can be approximated by the trajectories of a system which involves certain Lie brackets of 
the vector fields of the extremum seeking system. It turns out that the Lie bracket system directly 
reveals the optimizing behavior of the extremum seeking system. Specifically, we establish a the- 
oretical foundation and prove that uniform asymptotic stability of the Lie bracket system implies 
practical uniform asymptotic stability of the corresponding extremum seeking system. We use the 
established results in order to prove local and semi-global practical uniform asymptotic stability 
of the extrema of a certain map for multi-agent extremum seeking systems. 



1. Introduction 

In diverse engineering applications one faces the problem of finding an extremum of a map 
without knowing its explicit analytic expression. Suppose, for example, one vehicle tries to mini- 
mize the distance to another vehicle. The only information available, is the distance to the other 
vehicle. Clearly, the distance does not provide a direction in which the vehicle has to move. How- 
ever, it is intuitively clear that one can obtain a direction by using multiple measurements of the 
distance. Extremum seeking feedback exploits this procedure in a systematic way and can be used 
for steering dynamical systems to the extremum of an unknown map. Extremum seeking has a 
long history and has found many applications to diverse problems in control and communications 
(see [T3] and references therein). 

In this paper, we provide a novel methodology to analyze extremum seeking systems which 
differs from commonly used techniques. Specifically, this work contains three main contributions. 

First, we provide a novel view on extremum seeking by identifying the sinusoidal perturbation 
in the extremum seeking system as artificial inputs and by writing it in a certain input-affine form. 
Based on this input-affine form, we derive an approximate system which captures the behavior 
of the trajectories of the original extremum seeking system. It turns out that the approximate 
system can be represented by certain Lie brackets of the vector fields in the extremum seeking 
system which we call Lie bracket system. The proposed methodology is essentially different from 
results in the existing literature (see e.g. [5], [H] and [26]). 

Second, we establish a theoretic foundation which is based on this novel viewpoint. We prove 
that the trajectories of a class of input-affine systems with certain inputs are approximated by the 
trajectories of the Lie bracket systems. Similar results concerning sinusoidal inputs are covered 
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in [TU] and were extended in jS], [TT] to the class of periodic inputs. In [21] and [25] convergence 
of trajectories of a class of input-affine systems to the trajectories of more general Lie bracket 
systems was established. These results are closely related to our results. Furthermore, we prove 
under mild assumptions that semi-global (local) practical uniform asymptotic stability of a class of 
input-affine systems follows from global (local) uniform asymptotic stability of the corresponding 
Lie bracket systems. These results are based on [TS] and [T5]. Summarizing, to the authors best 
knowledge, the generality of the setup proposed herein was not addressed in the literature before. 

Third, we apply the foregoing results to analyze the behavior and the stability properties of 
extremum seeking vehicles with single-integrator and unicycle dynamics and with static maps. 
We formulate a multi-agent setup consisting of extremum seeking systems where the individual 
nonlinear maps of the agents satisfy a certain relationship which assures the existence of a potential 
function. We use the established theoretical results to show that the set of extrema of the potential 
function is (locally or semi-globally) practically uniformly asymptotically stable for the multi- 
agent system. This multi-agent setup is strongly related to game theory and potential games 
(see 14J). In the single-agent case, this potential function coincides with the individual nonlinear 
map. Similar extremum seeking vehicles were analyzed in |29j and 30 by using averaging theory 
(see [7] and [H]). The authors proposed various extremum seeking feedbacks for different vehicle 
dynamics and provided a local stability analysis for quadratic maps. Using sinusoidal perturbations 
with vanishing gains, the authors of [22] and [23] were able to extend these results to prove 
almost sure convergence in the case of noisy measurements of the map. In a slightly different 
setup the authors of [25] considered feedbacks which stabilize the extremum of a scalar, dynamic 
input-output map and established semi-global practical stability of the overall system under some 
technical assumptions. Multi-agent extremum seeking setups which use similar game-theoretic 
approaches can be found in [20] and [21], where the agents seek a Nash equilibrium (see [P7]). 
The authors proved almost sure convergence of the scheme but without explicit consideration of 
the global stability properties. A closely related result, which considers the local stability of Nash 
equilibrium seeking systems, can be found in [4]. 

Preliminary results of this paper were published in [2] and [3]. 

1.1. Organization 

The remainder of this paper is structured as follows. In Section 2 we illustrate the main idea 
using a simple example. In Section 3 we present theoretical results which relate the stability 
properties of an input-affine system to its Lie bracket system. In Section 4 we apply these results 
to analyze stability properties of some typical multi-agent extremum seeking systems. Finally, in 
Section 5 we illustrate the results with examples and give a conclusion in Section 6. 

1.2. Notation 

No denotes the set of positive integers including zero. Q++ denotes the set of positive rational 
numbers. The intervals of real number are denoted by (a, b) = {x £ R : a < x < b}, [a, b) = {x G 
R : a < x < b} and [a, b] = {x € R : a < x < b}. Let / : R" x R m ->■ R k , then we write /(•, y) 
if we consider / as a function of the first argument only and for all y G H. m . We denote by C n 
with n € No the set of n times continuously differentiable functions and by C°° the set of smooth 
function. The norm | • | denotes the Euclidian norm. The Jacobian of a continuously differentiable 
function b e C' 1 : R n ->• R m is denoted by 
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and the gradient of a continuously differentiable function 

i T 



dJ(x) d.J(x 
dx-i ' * " ' 5 dx n 



The Lie bracket of two vec- 



J E C 1 : R™ ->• R is denoted by V x J(a:) := 
tor fields f,g : M.x 1" — > 1" with f(t,-), g(t,-) being continuously differentiable is defined by 
[f,g](t,x) := ^^-f(t,x) - ^^-g(t,x). The a-neighborhood of a set S C K" with a € (0,oo) 
is denoted by uf :— {x e R n : inf. y(E 5 \x — y\ < a}. Uf denotes the closure of Uf . A function 
u : R — > R is called measurable if it is Lebesgue-measurable. We use s € C for the complex 
variable of the Laplace transformation if not indicated otherwise. 

2. Main Idea 

One simple extremum seeking feedback for static maps is shown in Fig. [I] (see also [5] and 
[30 ). Suppose that the function / g C 2 : R — > R admits a local, strict maximum at a;* and 
a, lj € (0, oo). 



- f(x) 





* — a \/lo cc 



tv^cos(wt) * — \/a;sin(wt) 

Figure 1: Basic extremum seeking system 

The overall system can be written as 

x = cty/uj cos(wt) + f(x)\fuisuY{tjjt). (1) 

The main idea is to identify sin(wi) and cos(W) as artificial inputs, i.e. U\(u)t) :— cos(cji) and 
u 2 (o;t) := sin(oji). Thus, we obtain an input-affine system of the form 

X = bi(x)y/UJUi(ut) + b2(x)\^UJU2(ujt) (2) 

with b\{x) = a and b%(x) = f(x). Interestingly, if one computes the so called Lie bracket system 
involving [61,62], i.e. 

i =-[&!, 6 2 ](z) = aV/(*), (3) 

one sees that this system maximizes f(z). Having in mind, that trajectories resulting from sinu- 
soidal inputs in (JlJ can be approximated by trajectories of ^ (see [5], [10], [11], [25]) allows us 
to establish a novel methodology to analyze extremum seeking systems. 

The goal of this paper is to generalize this viewpoint to a larger class of extremum seeking 
systems. We derive a methodology which allows to analyze a broad class of extremum seeking 
systems by calculating their respective Lie bracket systems. The procedure can be summarized as 
follows: Write the extremum seeking system in input-affine form, calculate its corresponding Lie 
bracket system, prove its asymptotic stability and conclude practical asymptotic stability for the 
extremum seeking system. 

3. Lie Bracket Approximation for a Class of Input- Affine Systems 

In this section we consider a class of input-affine systems depending on a parameter and deliver 
general results for approximating the trajectories of such systems by the trajectories of their 
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respective Lie bracket systems. First, we state the definition of practical stability of a compact, 
invariant set for this class of systems. Second, we prove that their trajectories are approximated 
by the trajectories of their corresponding Lie bracket system for large values of the parameter. 
Third, we show how the stability properties of the input-afhne system and the Lie bracket system 
are related. The results in this section, rely on a combination of results in [5], [TU], [TT], [25] and 

3.1. Practical Stability 

In the following, we define the notion of practical stability which is closely related to Lyapunov 
stability and applies to differential equations depending on a parameter. Throughout the paper, 
we denote this parameter as w. For related literature on this concept we refer to [15], [2B], [27] 
and references therein. 

Let x(t) := x(t; to, xq, uj) denote the solution of the differential equation 

± = f u (t,x) (4) 
through x(t ) = x , where the vector field / u :Ix R™ —> R™ depends on uj £ (0,oo). 

Definition 1. A compact set S C R™ is said to be practically uniformly stable for Q if for 

every e G (0,oo) there exists a 5 G (0, oo) and ujq G (0,oo) such that for all to £ R and for all 

u e (ojo, oo) 

x(t ) euf =>x{t) euf,t e [t ,oc). (5) 

Definition 2. Let 5 G (0,oo). A compact set S C R™ is said to oe 6 -practically uniformly 
attractive for Q i//or every e G (0, oo) t/iere exists atf £ [0, oo) and ujq suc/i t/iat /or a^/ to £ R 
and a/Z uj £ (wo, oo) 

x(t ) eUs ^ x(t) eUf,te [t + t f ,oo). (6) 

Definition 3. ^4 compact set S C R™ is said to be locally practically uniformly asymptoti- 
cally stable for ^ if it is practically uniformly stable and there exists a S £ (0,oo) such that it 
is 5-practically uniformly attractive. 

Definition 4. Let S C R™ be a compact set. The solutions of Q are said to be practically 

uniformly bounded if for every 6 £ (0,oo) there exists an e € (0, oo) and ujq £ (0, oo) such that 
for all to £ R and for all uj £ (wo, oo) 

x(t ) £ ^x(t) €Uf,t G [t ,oo). (7) 

Definition 5. ^4 compact set S C R" is said to 6e semi-globally practically uniformly 
asymptotically stable for Q i/ it is practically uniformly stable and for every S G (0,oo) it 
is S -practically uniformly attractive. Furthermore the solutions of Q must be practically uni- 
formly bounded. 

For the notion of stability in the sense of Lyapunov, we refer to e.g. [7J, [15] and [18] , 

3.2. Lie Bracket Approximation 

Throughout the paper, we consider the class of input-afhne systems which can be written in 
the following form 

m 

x = b (t,x) + b t (t, x) \fuluj (t , wt) (8) 
i=i 
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with x(to) — xq £ R™ and lj £ (0, oo). Next, we define a differential equation, which we call the 
Lie bracket system corresponding to ([8| 

m 

z = b (t,z) + ^2 [ b i,bj](t,z)vji(t) , Q s 

i=l 
j=i+l 



with 



T 



Vji( t ) = 7F I %M) / Ui{t,T)drdB. (10) 



Remark 1. If Ui can be decomposed as Ui{t,uit) = ri(t)ui(ujt) , i = 1, ... ,m, then ([8| yields x = 
b (t,x) 

+ ^ i _ 1 bi(t,x)T/ojUi(u)t) with b{(t,x) — bi(t,x)ri(t). Hence, in this case explicit dependence of 
Ui on the first argument can be omitted, which is usually done in the existing literature (see e.g. 

We impose the following assumptions on hi and uf. 
Al i,eC 2 :Ix R n -)• R n , i = 0, . . . , m. 

A2 For every compact set JC C R" there exist Ai,...,A$ £ [0, oo) such that \bi(t, x)\ < A\, 
\ dJ %^\ < A 2 , < A 3 , < A A , < A 5 , < Ae for a u 

cc G /C, f G M, i = 0, . . . , m, j = 1, . . . , m, k = j, . . . , m. 

A3 Ui : K x R — > M, i = 1, . . . , m are measurable functions. Moreover, for every i = 1, . . . , m there 
exist constants Li, Mi £ (0, oo) such that \ui(ti,9) — Ui(t2,9)\ < L^ti — t 2 \ for all ti,t 2 £ R 
and such that sup t egR #)| < Mi. 

A4 Ui(t,-) is T-periodic, i.e. Ui(t,8 + T) = Ui(t,9), and has zero average, i.e. Ui(t,r)dT = 0, 
with T e (0, oo) for all t, G R, i = I, . . . , m. 

Remark 2. Assumption Al states a regularity assumption on the vector fields, which are usually 
assumed to be smooth in the case of extremum seeking systems, see f§jj and )2b^ . 

Remark 3. Assumption A2 states that expressions involving b t , i — 0, . . . , m and their derivatives 
must be bounded uniformly int. A similar assumption was made in Eq. (2.2), Section 2 in UP}/ . 

Remark 4. Assumption A3 imposes measurability on u i: i = l,...,m which is necessary to 
establish existence of solutions of ^ (see Theorem^ in Appendix \Appendix A ). Alternatively, 
one could have imposed that the inputs Ui, i = 1, . . . , m are continuous functions and argue using 
the existence and uniqueness theorem of Picard-Lindeldf (see \T^). However, this does not cover 
the case of piecewise continuous inputs, which might be interesting for further applications, i. e. 
replacing the sinusoids with piecewise constant functions in the extremum seeking systems. 

Remark 5. Similarly as in J^j/ we impose in Assumption A4 the T -periodicity and zero average 
of Ui, i = l,...,m, which is common in the averaging literature but also in the literature dealing 
with Lie brackets. 

Finally, we introduce the set B of initial conditions for (|9| which have uniformly bounded 
solutions, i.e. 

B = {z £ R" : there exists an A £ (0, oo) 

such that for all t a £R: \z(t)\ < A, (11) 
t £ [t ,oo) with z(t ) = z}. 
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The set B is used in the proof of the following theorems and is crucial in order to assure existence 
of trajectories uniformly in to. 

In the following, we state the main theorems which relate stability properties of the systems 
in pi) and Q. First, we state that trajectories of Q are approximated by trajectories of (|9| 
formulated in the sense of Hypothesis 2 in [To] . Related results are also presented in [5], [Ly 
and [27! . However, we show for a larger class of inputs that the time interval can be made 
arbitrary large by choosing lj sufficiently large. Similar as in |15j , we extend this result to infinite 
time-intervals and prove that the semi-global (local) practical uniform asymptotic stability of the 
input-affine system ^ directly follows from the global (local) uniform asymptotic stability of the 
corresponding Lie bracket system ([9]). 

Theorem 1. Let Assumptions y^J^J^ be satisfied. Then for every compact set K. C B, for every 
D G (0,oo) and for every tf G (0,oo), there exists an ujq G (0,oo) such that for every lo G (cjo,oo), 
for every to G K and every xq G K. there exist solutions x, z : K — > R™ of Q and ^ through 
x(t(j) = z(to) = xq which satisfy 

\x(t) — z(t)\ < D, t G [to, to + tf]. (12) 

The proof of Theorem [l] goes along similar lines as indicated in B.3, p. 1941 in [TB] but 
considers the more general case of inputs, which are characterized by Assumptions A3 and A4. It 
can be found in Appendix |Appcndix C| 

Theorem 2. Let Assumptions J^-A^ be satisfied and suppose that a compact set S is locally 
uniformly asymptotically stable for (pM) . Then S is locally practically uniformly asymptotically 
stable for ^ . 

The proof can be found in Appendix |Appendix D| 

Theorem 3. Let Assumptions ^fTl-j^ be satisfied and suppose that a compact set S is globally uni- 
formly asymptotically stable for Q . Then S is semi-globally practically uniformly asymptotically 
stable for (|Sj) . 

We omit the proof of Theorem [3] since it is already covered in [T3] for the case of S being the 
origin. The proof can be directly applied here by replacing the Euclidian norm with the distance 
to the set S. 

Remark 6. The results above only capture stability and not performance and do not deliver a 
systematic way for choosing uj. The notion of practical stability only requires the existence of ujq 
without explicitly considering a specific value. As indicated by Theorem^the choice of oj depends 
on the set of initial conditions K,, the distance D and the time tf. 

This is the basis for the result of the next section, which generalizes the ideas of the example 
in Section 2. 

4. Lie Bracket Approximation of Extremum Seeking Systems 

In this section, we show how the results from the previous section can be applied to multi-agent 
extremum seeking systems. As indicated in Section 2, the procedure is to write the extremum 
seeking system in the input-affine form, calculate the corresponding Lie bracket system and con- 
clude using Theorems [2] and [3] the respective stability properties from the stability properties of 
the Lie bracket system. 
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In the following, we define a suitable framework for multi-agent extremum seeking systems. 
Given is a group of N agents in the plane, which are meant to achieve a common goal formalized 
as seeking an extremum of a common map F. Specifically, let us enumerate the agents using the 
superscript i. The position of agent i is denoted by x l — [x\,x\\ T £ M 2 . We define furthermore 
x := [x\, x\, . . . , Xi , x%] as the position vector of the overall system. Every agent is equipped 
with a specific extremum seeking feedback, which is defined below. We do not assume that all 
agents are seeking the extremum of the same map, but rather that each agent is equipped with an 
individual map p : R 2N — > R, i = 1, . . . , N, which also depends on the states of the other agents 
and satisfies 

Bl /' £ C 2 , i = 1, . . . , N. 
Furthermore, the individual maps have to satisfy the following assumption 

B2 There exists a function F £ C 1 : R 2N -> R such that V S if(x) = V S iF(x), i = 1, . . . , N, x £ 
R 2N . 

These conditions implies, that if every agent moves into the direction of the gradient of its indi- 
vidual map P then it also moves in the direction of the gradient of F. We call this a potential 
function. The goal of the multi-agent system is to find the minimum (maximum) of the common 
map F by only seeking the minimum (maximum) of the individual map 

The following assumptions guarantee the existence of local (global) maxima of the potential 
function 

B3 There exists a nonempty and compact set Si oc C R 2N of strict local maxima and a 5 £ (0, 00) 
such that F(x*) > F(x) for all x* £ Si oc and all x £ Wf ioc \<S ioc . Furthermore, V s F(x) = 
implies x £ Si oc for all x £ Uf loc , 

B4 There exists a nonempty and compact set S g i b = {x £ R 2N : x — arg max xeK 2N F(x)} of 
global maxima. Furthermore, F(x) — ¥ —00 for \x\ — > 00 and V ' s F(x) = implies x £ S g i b 
for all x £ R 2N . 

This framework originates from game theory, where Assumption B2 formally defines a potential 
game with potential function F. We refer to [T3] for an introduction to potential games. 

Remark 7. Under the assumptions above, the common goal can be formalized as the minimization 
(maximization) of the potential function F. There exist powerful tools to construct meaningful 
individual maps for a given potential function (see e.g. the approach using the so-called Wonderful 
Life Utility in ]28p. The design should be done such that an optimization of the individual maps 
leads to an optimization of F , see For this case, even though the utility functions are designed, 
they usually depend on some parameters or functions (e.g. environmental conditions, individual 
agents' properties) which are unknown a priori. A typical example for this scenario is the coverage 
control problem formulated as a potential game in ]12j and J1J/. These aspects justify the usage of 
extremum seeking in this setup. For a specific application of the extremum seeking in a potential 
game framework we refer to JIJ/. 

In the next subsection, we show how the framework above can be combined with extremum 
seeking agents. We saw in Section 2 that the trajectories of the extremum seeking system can be 
approximated by the trajectories of its corresponding Lie bracket system, which moves into the 
gradient direction of its individual map. We generalize this to the multi-agent case. If each agent 
is equipped with an extremum seeking feedback which drives it into the gradient direction of its 
individual map p, we expect with Assumption B2 that the overall system practically converges 
to an extremum of F. This is shown in the next subsection. 
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Figure 2: Singe-Integrator Dynamics 



4--1- Multi-Agent Extremum Seeking 

In the following, we show how extremum seeking can be applied to the above framework 
assuming single-integrator agent dynamics. 

Consider the system in Fig. [2] which is motivated by a similar extremum seeking feedback 
in |30j . Since the agents move in the plane, there are two extremum seeking loops, one for each 
dimension. The perturbations are chosen to be sinusoidal, whose frequencies are chosen for each 
agent individually, as specified below. The high-pass filters G l (s) = j^j, i = 1,...,N are 
introduced since they provide better transition behavior by removing possible constant offsets of 
the individual maps /*, i = 1, . . . , m. They introduce an additionally degree of freedom, but do 
not influence the stability of the overall system, as it can be seen in the proof of the following 
theorems. 



Define x e 



and x := [ 



with x l e denoting the state of the filter G l {s) 
■^Tf-t, i.e. in state space form we have x l e = —x\h l + u l and y l = —x\h l + u l with u 1 — f l (x). 
The differential equations describing the dynamics of agent i are given by 



x\ =c l (f(x) - x % e h*) Vw*ul(w*t) + aVw*uJ(w*t) 
x l 2=- c l {f{x) - x\W)^u\{uj l t) + a'V^uKuH) 

±i = -x i e h i +f\x) 



(13) 



with u\(uj l t) = sin(w^), u l 2 (u)' t t) = cos(o/t). 

We need an additional assumption for the multi- agent case concerning the parameter uj. We 
see in the proof of the next theorem that if the following assumption is satisfied, then some of the 



Vji in ( 10 ) vanish in the corresponding Lie bracket system. This can be assured by assuming 



B5 u/ = a l uj and a 1 ^ a? ,i ^ j, a 1 £ Q++, ul £ (0, oo), h l ,a l ,c z £ (0, oo), i,j = l,...,N. 

Since the high-pass filter -^p- introduces an additional state x l e , which has also to be taken into 
account in the analysis, we denote by 



£ s := {x e £ R N : 
, r/ X (2) 



fM lT 

h N J 



(14) 
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with S is either iS; oe or S g i b, the set which is shown to be attractive for the filter states x\, 
i = l,...,N. 

Theorem 4. Consider a multi-agent system with N agents, each one having dynamics given by 
(13). Let Assumptions Bl to B3 and B5 be satisfied, then the set Si oc x £ Sl °' is locally practically 
uniformly asymptotically stable for the overall system with state [x T , xJ] T . 

Proof. The proof can be split up into three steps. In the first step, we rewrite the system in the 
input-affine form. In the second step, we calculate the corresponding Lie bracket system and in 
the third step, we prove uniform asymptotic stability of the Lie bracket system. Theorem [2] then 
allows to conclude practical asymptotic stability for the original system. 

In the first step, we rewrite the overall system with state x = [x T , ij] 1 ^, where each component 
is described by the differential equations given in (13), as input-affine system of the form 

N 

x = y^ bi(x) + b{(x)Vo? sin(cj^) 

=:u|(u) 4 t) (15) 

+ b\ (x) VuS cos (ljH) 

=: u|(wn) 

with &q,6^,&2 having non-zero entries only at positions corresponding to agent i and zeros else- 
where, i.e. 6j(as) = [0, . . . , 0, -xi+f* (x), 0, . . . , 0] T , 6* (a;) = [0, . . . , 0, ^(^(Sj-ijV), a\ 0, 0, . . . , 0] T , 
b\{x) = [0, . . . , 0, a\ -c*(f(x) - xih*), 0, 0, . . . , 0] T . 

Note that due to Assumption B5 we have that a.; can be written as a,; = |f with Pi,qi € N and 

define q := YliLi qi and u = ^. Thus, a^o; = = pi Ylj^i — i = 1, . . . , N and j = 1, 2 
and for ri 1 :— p. L Yij^i Qj G N. We rewrite ( 15 1 as follows 

JV 

x = bl(x) + b\(x)\fn l \/Cju\(n l Lbt) ^ 
+ bl(x)VriVtiu l 2 {n l u)t). 

It can directly be seen that u l k (n l 9) £ {sm(n l 8),cos(n l 9)} are also 27r-periodic in n l Qt for i = 
1, . . . , N and k = 1, 2 and for n % e N. 

In the second step, we calculate the corresponding Lie bracket system as defined in Define 
z := \z\,z\, . . . , zf , z?] T , z e := [z\,..., zf] T and z := [z T , zJ] T and = i / 2Tr u l fc (nV) /J" ^ ( n ' 
which are constant for all i, j = 1, . . . , N and k, I — 1, 2. 

The crucial point now is that some Lie bracket in the differential equation of the overall system 
vanish du e to the choic e of different parameters of for the agents. We obtain using Lemma [l] (see 
Appendix Appendix B) that v\l\ = —^r f° r au n% — n ^ ancl & = I an d = otherwise. Thus, 
the Lie bracket system simplifies to 

N 



^bUz) - ^-[Vn~%,Vn~%](z) 



i=l 
N 



(17) 
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Explicitly, for the states of agent i we obtain 



4 =±(cWV 4 f(z) - j\ 4 f{z)(f{z) - zfrj) 
A =\{jc?V A f{z) + e\ z{ f{z){f{z) - zlh)) 

zl = -zih i +f i {z). (18) 



In the third step, we prove uniform asymptotic stability of the set Si oc x £ 5i °<= for (17 1. We 



first need to show existence of the solutions of (17 1 on [t , oo) for all t Q <E E. Note that the vector 



field in (17) is independent of t and continuously differentiable in z. The existence and uniqueness 
theorem by Picard-Lindelof (see pQ) guarantees that there exist a time tf € (0, oo) and a solution 
of z : E -> R 3N defined on [t ,t + t } ) for all £ € E. Note furthermore, that with h* € (0,oo) 
in Assumption B5, the differential equation for z % R , i.e. z\ — —h l z l e + u with u = f l {z) in (JTsJ) is 
linear and its origin is exponentially stable for u = 0. Thus if f l (z(t)) is bounded then z*(£) exists 
and is bounded with gain A- for all i = 1, . . . , N, for all to G E and for all t € [£o, °°)- Suppose 
now that Si oc is uniformly asymptotically stable for z, then it can be shown that the set £ loc is 
uniformly asymptotically stable for Zg, i = 1, . . . , N. Therefore, the set Si oc x £ Sl °" is uniformly 
asymptotically stable for the overall system [z T , zJ] T . 

It is left to show that the set Si oc is uniformly asymptotically stable for z. Choose V := 
—F which is due to Assumption B3 a valid Lyapunov function in Uf 1 "". Observe that due to 
Assumption B2 we have that V^/^z) = W Z iF(z), i = 1, . . . , N and thus 



N i i 
C rot 



2 , 

*=i v (19) 



V = -J2 — (V z jF(z) T V z; F(z) 

+ V z ,F(z) T V zi F(z) 

Due to c 1 , a 1 £ (0, oo), i = 1, . . . , N in Assumption B5, we know that V(z(t)) is decreasing along 
the trajectories of z(t) for all z(<o) € ^f !oc j all € K and all t € [io^o + ^/)- We conclude that 
\z(t)\ is bounded and therefore all f l (z(t)), i = 1, ...,N, are bounded for all z(t ) € Uf loa , all 
t Q e K and all £ e [t ,oo). Thus, z(£) = [z(£) T , z e (£) T ] T exists for all £ € E, for all z(t ) € Wf !oc 
and for all £ €E [£q,oo). Furthermore, we conclude with (19) and Assumption B3 that the set Si oc 



is locally uniformly asymptotically stable for the subsystem z = \z{,zl,...,z™z$] T in dl7b. 

Note that due to Assumption Bl and the fact that u l k (n l 9) g {sin(n z 6*), cos(n l 6')} for i — 
1, . . . , N and fe = 1,2 we conclude that Assumptions A[l]to AH] are satisfied. Thus, with Theorem 
[5] the set Sioc x E Sloa is locally practically uniformly asymptotically stable for the overall system 
with state [x^xj] 1- . □ 

Theorem 5. Consider a multi-agent system with N agents, each one having dynamics given by 
(13). Let Assumptions Bl, B2, B4 and B5 be satisfied, then the set Si oc x £ s s'°>> is semi-globally 
practically uniformly asymptotically stable for the overall system with state [i T ,i 6 T] T . 

Proof. If Assumption B4 is satisfied then S g i b is a connected set containing the global maximum 



of F. Furthermore, F is radially unbounded and with (19) we see that if V(z) = implies 
z(t) £ S global- Thus, we conclude that <S; oc x £ s s'°>> is globally uniformly asymptotically stable for 
(17) and thus with Theorem [3j it is semi-globally practically uniformly asymptotically stable for 



the overall system with state [x , x e ] . □ 
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In the following, we analyze the same setup but replace the single-integrator dynamics with 
unicycle dynamics as shown in Fig. [3j The setup is motivated by [29] . 







Unicycle 


4 







X ] ,j i 



s+h' 



q' v u l cos(u't) 1 V Lj l sin(w*t) 

Figure 3: Unicycle Dynamics 
Let us consider the unicycle model for each agent given by the equations 
x\ = u l cos(xg), x\ — u 1 sin(a;g), x\ = v 1 . 



(20) 



The extremum seeking feedback controls only the forward velocity of the vehicle, whereas the angu- 
lar velocity is constant, so that the inputs to each vehicle are u l (t, x) = (c % (p{x)—x\h % )\Z^sm.(uj l i)- 
ttW cos(a/t)) and v l = ft\ We assume that Xg(to) = and for all i = 1, . . . , N and 



B6 ft 1 = <f ft with d i G 



ft G K\{0}. 



Remark 8. It becomes clear in the proof that the corresponding vector field of Lie bracket system 
is time-varying and vanishes at discrete points in time. Assumption B6 assures that the vector 
field is periodic, so that a LaSalle-like argument can be used in order to prove uniform asymptotic 
stability. Note, that the ft 1 's can be equal, whereas the uj 1 's must be different for all agents. 



By substituting the expressions for the inputs into (20 1 and replacing x\(t) — ft'i we obtain 

x\ = (c^f^xyxih^ViJuiiujH) 

+ a l \fl7 l ul l (uj t t) ) cos(ft^) 



c , (f(5)-i^ , )Vw i u' 1 (w , t) 



(21) 



+ aVw'ti , 2 (w'() ) sin(ft l i) 
K = ~ x lhi + f(x) 
with u\(uj l t) = s'm(Lo l t), u\{u)' t t) = cos(a/t). 
Theorem 6. Consider a multi-agent system with N agents, each one having dynamics given 



by (21). Let Assumptions Bl to B3, B5 and B6 be satisfied, then the set Si oc x £ s '° c is locally 
practically uniformly asymptotically stable for the overall system with state [x T ,xJ] T . 
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Proof. The proof goes along the same lines as the proof of Theorem|4] In the first step, we rewrite 
the overall system as input-affine system 



jV 



= Y^ b U x ) + b\(t,x)Vu} i u{{uH) 
+ b 2 (t,x)VJu 2 {cj l t) 



(22) 



where bQ,b\,b 2 have non-zero entries only at the positions corresponding to agent i and ze- 
ros elsewhere, i.e. b^(x) = [0, . . . , 0, 0, -x\hi + f(x), 0, . . . , 0] T , b\(t,x) = [0,..., (c\p{x) - 
x l e /i l ))cos(fi^),(c l (/ 4 (x) - x\K 1 )) sin(fTt), 0,0, ...,0] T and b 2 (t,x) = [0, . . . , a 1 cos(fTf), 
a*sin(m), 0,0,..., 0] T . 



Note that due to Assumption B5 eij can be written as dj = ^ with Pi,^ G N and define 
g := YiiLi Qi an< ^ ^ = Thus, aiUj — ^ui — pi Ylj^i = i = 1, . . . ,N and j = 1,2 and for 



n * := K Ilj^j ?j e N. We rewrite (22) as follows 

N 

x = ^2b l (x) + b\(t, x)Vn i Vtiu l 1 (n l £jt) 



i=l 



(23) 



+ b\{t, x)V nW wu\{n % G)t) . 
In the second step, we calculate the corresponding Lie bracket system as it was defined in ([9]), 

N 1 

* = £to-o[M(*,*)- (24) 



By the same reasoning as in the proof of Theorem |4j this yields for the state of agent i 

+ jtfVzi.fiz) cos(fT<) sin(O l i))) 

i|=i(c < a i V^/ i (2)sin 2 (n i t) ( 25 ) 
+ cVV^f (z) caa(SlH) sin(ff ; i))) 

4 = -4fc* + /'(*)■ 

In the third step, we prove uniform asymptotic stability of the set Si oc x £, Sl °" for the Lie 
bracket system of (23). Due to Assumption B3 we exploit the function V := — F as a Lyapunov 
function candidate which is valid in Uf loc . Observe that due to Assumption B2 we have that 
V s *P{z) = V E iF(z),i= 1,. .. ,N and thus 



V = - 



(V z ,F(z) cos(ft't) + V,,F(*) sin(O^)). 



(26) 



We have that c l ,a l € (0,oo), i = 1, ...,N from Assumption B5, and thus V is negative semi- 
definite. Observe that the vector field in (25) is time- varying and there are time-instances where 
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z(t) = 0, but which are not steady-states for the system. Next, we make use of Assumption B6, 
which assures the existence of k\ l % £ N, i = 1, . . . , N such that d % — 4-. One can verify that the 



vector field of the overall system ( 24 ) consisting of N agents with system equations as in ( 25 



is 



T-periodic with T = ^ Jlili We 

can now use Theorem 1.3, p. 50 in [T5] which is a LaSalle-like 

argument to conclude uniform asymptotic stability for periodic vector fields. It is left to show 



that no trajectory of (24) can stay identically in the set where V(z) = except for z £ Si oc . 
To see this, observe that the summands of V can only be equal to zero if W z iF(z(t)) cos(il l t) + 
V z i F(z(t)) sin(i7 l i) = 0, i — 1,...,N. On the set V(z) = the differential equation yields 
z\ — z\ — and therefore z\(t) = const, and z l 2 (t) = const.. Thus V z iF(z(t)) = const, and 
V z iF(z(t))) = const.. But there are no constants a, b £ K such that acos(f2 4 i) + bsin(£l l t) = 
for all t £ [io>oo) except a = b = and therefore V z iF(z{t)) = V z iF(z{t)) = 0. We conclude 



that the set Si oc is locally uniformly asymptotically stable for the subsystem z in ( 25 1 . Observe 



furthermore, that due to h l £ (0, oo) in Assumption B5, the differential equation for z % e , i.e. 



= —h l z\ + u with u = p{z) in (181 is linear and its origin is exponentially stable for u = 0. 
Thus if P(z(t)) is bounded then z\{t) exists and is bounded with gain -h for alH = 1, . . . , N, for 
all to € K and for all t £ [to, oo). Therefore, the set Si oc x E Sl ° c is uniformly asymptotically stable 
for the overall system [z T , zJ] T . 

Note that due to Assumption Bl and the fact that u l k (n l 9) £ {sin(n J 0), cos(n l 6')} for i = 
L . . . ,N and k = 1, 2 we conclude that Assumptions A[l]to A[4]are satisfied. Thus, with Theorem 
[2] the set Sioc x E Sloa is locally practically uniformly asymptotically stable for the overall system 
with state [x^xj] 1- . □ 

Theorem 7. Consider a multi-agent system with N agents, each one having dynamics given by 
(21). Let Assumptions Bl, B2 and B4 to B6 be satisfied, then the set Si oc x £ S 9'°>> is semi-gobally 



practically uniformly asymptotically stable for the overall system with state [x , xj ] . 

The proof uses the same argumentation as the proof of Theorem [5] 

Remark 9. It is important to observe that the amplitudes and frequencies of the sinusoids of the 
extremum seeking feedbacks in Fig. [2] and Fig. [5] are different, compared to the amplitudes in 
the corresponding schemes in the existing literature \3U§ , \29j and [26]. The choice of t/uj for the 
amplitudes in combination with uj for the frequency is crucial in order to obtain the Lie bracket 
system as approximation of the input-affine system. This is also pointed out on p. 241 in flOf . 

Comparing Theorem^to Theorem 10.5 on p. 4II in we see that there is a close relationship 
between the results herein and averaging theory. The Lie bracket system in ^ can be seen as the 
average system of Q (see 111]). In J30f and \2ty the amplitudes of the perturbations are chosen to 
be lu and 1, respectively, whereas the frequencies are chosen to beu. Even though the schemes differ 
only in the choice of the amplitudes, the observation above let us expect that the average systems 
of the corresponding extremum seeking systems in 130}/ and [29] differ from the Lie bracket systems 
obtained here. A similar reasoning applies to [26] concerning the results on static maps, where the 
parameters do not influence the frequencies of the perturbations but only their amplitudes. 

For results concerning averaging theory and practical stability, we refer to \27\j and references 
therein. 

Remark 10. Theorem^ and Theorem^ state local and semi-global practical uniform asymptotic 
stability for a group of N agents with single-integrator and unicycle dynamics. A special case is 
a single-agent extremum seeking system for which we have N = 1 and f 1 = F. Furthermore, a 
similar analysis can be adopted in a straight forward fashion to the case of extremum seeking in 
one dimension by removing one feedback loop in Fig. [1[ 
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Remark 11. In the presented schemes in Fig. [7| Fig. [#] and Fig. [5] it is not essential that the 
perturbation signals are sinusoidal. Theorem^ and Theorem^ can be applied to analogous schemes 
where the sinusoidal perturbations are replaced with other appropriately defined periodic signals 
as long as they satisfy Assumptions A3 and A4- This also includes discontinuous and/or non- 
differentiable signals such as square, triangle or sawtooth waveforms (see also Remark 4 above). 

In the following, we show further advantages of the Lie bracket approximation using numerical 
experiments. 



5. Examples 

In this section, we show numerical examples which illustrate the main results. First, we 



compare for different values of u> the trajectories of the single-integrator system of ( 15 ) with 



its corresponding Lie bracket system (17). Second, using the Lie bracket system, we explain a 
characteristic effects which we denote as characteristic points and which arise in the extremum 
seeking with unicycle dynamics. 

We consider a system of N — 3 agents and enumerate them with a, b, c. We assign each agent 
the following maps 



+ xf + e(~ x ° 2 - 10, 
/*(*)=- \{x\ + lf- l -{x h 2 + lf 



(27) 



2 y 1 ' 2" z ' (28) 
+ sm(x\ + x|) - 10, 

/ c (^)=-^i + l) 2 -^2-l) 2 + 10. (29) 

We choose the parameters h = h a = h b = h c = 1, a a = a b = a c = 1, c a = c b = c c = 0.3 and 
the initial conditions [xq,^] 1 ^ = [2, — 2, — 2, 2, — 1, 2.5, 0, 0, 0] T . Observe that each of the f l, s, 
i = a,b,c are functions of the states of the respective other agents. 
Furthermore, we consider the quadratic function 

F[x) = -^(x-x*) T Q(x~x*) (30) 

where x* = [1, 1, —1, —1, —1, 1] T and the diagonal matrix Q = diag(l, 1, 1, 1, 1,3). We can verify 
that V^i/^x) = V S iF{x), i — a, fe, c and we see that F is quadratic and attains its maximal value 

at x*. We expect from Theorems ^ and that [(x*) T , ^fp-, ^fp-, ^f 1 )Y is semi-globally 
practically uniformly asymptotically stable for the extremum seeking systems. 

In Fig. [4] the trajectories of the original and the Lie bracket systems are depicted with u = 10 
and uj a — w, uj b — 2uj, uj c — 3a;. The trajectories of the Lie bracket system captures the qualitative 
evolution of the trajectories of the original system. In Fig. [4] we see a simulation with the same 
parameters but with w = 100. 

These examples illustrate two properties. First, the trajectories of the original system approach 
those of the Lie bracket system for large values of uj. This observation points up the result of 
Theorem[l] Second, we deduce from Fig. |4]and Fig. [5]that even though each of the /*'s, i = a,b,c 
contains highly nonlinear terms depending on the states of the other agents, the overall system 
practically converges even for small values of uj to the expected extremum. 
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Figure 4: Comparison of trajectories of a three-agent single-integrator system and its respective 
Lie Bracket system, for uj = 10 




Figure 5: Comparison of trajectories of a three-agent single-integrator system and its respective 
Lie Bracket system, for ui = 100 
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Figure 6: Comparison of trajectories of a three-agent unicycle system and its respective Lie Bracket 
system, for oj = 80 



The same result can be observed in the case of unicycle dynamics and the same choice of 
parameters as above, with additionally fl a = 1, Sl b = 2, il c — 3. In Fig. [6] the trajectories of the 
original and the Lie bracket systems are depicted for ui = 80. Observe that the overall system 
practically converges as expected to the extremum. The trajectory of the extremum seeking 
system contains characteristic points, which also appear in the trajectory of the Lie bracket system. 
Apparently the vector field changes its direction abruptly. This can be explained by regarding the 
differential equation of the Lie bracket system in (251, which is time- varying and vanishes at the 
zero-crossing instances of the sinusoids. 



6. Conclusion 

In this work we developed a methodology, which led to a novel interpretation as well as to 
novel stability results for extremum seeking systems. By identifying the sinusoidal perturbations 
of the extremum seeking as artificial inputs, we were able to rewrite the system in a certain input- 
affine form and to relate this system to the so-called Lie bracket system, which nicely reveals the 
optimizing behavior of extremum seeking. The Lie bracket system viewpoint of extremum seeking 
allowed us to establish strong stability results for extremum seeking systems. We proved that the 
trajectories of systems belonging to a certain class of input-affine systems can be approximated 
by the trajectories of their corresponding Lie bracket system. Furthermore, we showed that global 
(local) uniform asymptotic stability of the Lie bracket system implies semi-global (local) practical 
uniform asymptotic stability of the input-affine system. We applied these results to a multi-agent 
extremum seeking system consisting of agents with either single-integrator or unicycle dynamics. 
Finally, the results are illustrated using numerical examples. 

Appendix A. Existence and Uniqueness 

Consider the differential equation 

x = f(t,x) (A.l) 



1G 



t n and with initial condition x(to) — xq £ W 1 . If there exist at e € (0, oo) and 
l n such that 



with / : M x : 
a function x 



x{t)=x(t a )+ / f(T,x(r))dT (A.2) 

J to 

and ±(i) = /(£, x(t)) for i £ [to, io + except on a set of measure zero, then x(-) = x(-; t , xo) is 
said to be a solution of ( A.l ) through x(t ) — x defined on [to, t + t e ). 



Theorem 8 (see [TJ Consider ( A.l ) and suppose for every compact sets TCR and X C 
there exist measurable junctions M, L : R — > R suc/i £/ia£ 



|/(t,a:)|<M(i), 
/(i,a:a)|<i(*)|a!i 



x 2 



(A.3) 



i G T,x,Xi,X2 £ X. Then for every to £ T and x(to) £ X there exist a t e £ (0, oo) and a unique 
solution x : R — > 1™ through x(t Q ), which is defined on [to, to + t e ). 



Appendix B. Preliminary Lemmas 
Lemma 1. Let 

with n l ,n 3 £ N, u l (n l t) £ {sin(n l t) , cos (n z t)} , then 



u\n l T) \ u j {n j 6)d6dn 



(B.l) 



l 

2n> 



1 

" 2n* 



n l = n j ,u l (n l t) — srn(n*f), 

v? (n't) — cQs(n J t) 
n l = n j ,u l (n l t) = cos(n l t), 

u 3 (n 3 t) = sminH) 



(B.2) 



^0 else . 

Proof. The result follows by a direct calculation. □ 

Lemma 2. Let u : R x R — > R satisfy Assumption Furthermore, u{t, •) is T -periodic, i.e. 
u{t,6 + T) = u(t,8) for some T £ (0, oo) and all t, 6 € R. Then, there exist k\,ki £ [0, oo) suc/i 



£/iai £/ie inequality 







[ «M)< 


26Mdr 


/o 





< 



fcl(t-*0) + *2 



(B.3) 



is satisfied for all (o£M and all t £ [to, oo). Furthermore, k 2 = if uj(t — t ) is an integer multiple 
ofT, i.e. there exists an n £ No smc/i i/iai w(t — to) = Tn. 

Proof. Using the fact that u(t,ujt) 

= Jq u(T,LUT)d9 and applying the change of variables r — lot, dr — ujdr, the expression in 
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the norm of left hand-side in (B.3I yields 



1 

Tuj Jo 



T Mi 



u(-,r)-u(-,6)drd0. 



(B.4) 



Since T G (0, oo) we can divide [coto, cut] into n £ No pieces of length T such that U)(t — to) = Tn + 5 
with < S < T being the leftover piece. We obtain for (B.4) 



n-l r T r u>t +T(k+l) 



1 " 

-Y 



To; 



fc=0 



Jbjt +Tk 



tt(~,r) - u(-,d)drdd 

UJ UJ 



Ri, 



where we introduced the left-over piece 

1 



Ri := 



Tuj 



T rut +Tn+S 

/ u(-,r)-u{-,6)drd6, 

Jut +Tn u w 



(B.5) 



(B.6) 



which is considered later. 

The integration interval in (B.5) is now shifted by introducing the change of variable s = 
r — Luto — Tk, ds = dr 



i " * 
Tuj ^ 

k=0 

+ Ri. 



n—l r T rT 



JO 



r h k(s) , , ,h k {s) 
u{ , hf.(s)) ~ u{ 7 U)dsd& 

UJ UJ 



(B.7) 



with /ifc(s) := s+uto+Tk. Since u(t,-) is T-periodic, it follows that u{ t ^f L 1 hk(s)) — u( hk ^ s \ ho(s)). 
Thus, this simplifies to 



n— 1 r T rT 



Tuj 



k=0 

Ri 



JO 



u( k ^ - , ho(s)) — u( k ^ - , 6)dsd6 

UJ UJ 



(B. 



Note, that since the integration with respect to s and with respect to 9 is performed from to T 
and due to the periodicity of u(t, •), we can add J Q T u( f ^,6)d6 and subtract J Q T u(^) , h {s))ds 
which sums up to zero. We obtain 



n— 1 r T rT 



Tuj 



k=0 



, h k(s) . ,h k (0) 
u{ ,ho{s)) - u{ ,h [s)) 

UJ UJ 



+ u (Mf) } q) _ u (^A^)dsd6 + r x . 

UJ UJ 



(B.9) 



Assumption A[3] yields the existence of L € (0, oo) such that the above expression can be bounded 
from above as follows \u(^±, h (s)) -u(^, h (s))\ < %\s\ and \u(^, 6) -u(^±, 8)\ < 
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Thus, (|B.9|) can be upper bounded by 

n—X pT pT 

Tu) 



X -Y. I I i-\ s \dsde + \Rx\ 

u f~^Jo Jo w 



k=0 J ° J ° 

^n+|iZi|. (B.10) 



We now consider the expression R\ in (B.6|. Assumption A[3j yields the existence of M € (0,oo) 
such that it can be upper bounded as follows 

\Ri\<~h / 2Mdrd0= — . (B.ll) 

Therefore, using the definition of n = CJ ( t ~j i °)~ l? we obtain 

T 2 L 2MS T 2 L uj(t -t )-S 2MS 
— n+ = — ^ + 

^ TL(t - t ) + 2M5 { ' ' 

u 

Choosing k\ :— TL and k<z := 2MS proves the first claim. If ui(t — to) = Tn then 5 = and 

therefore, k 2 = which proves the second claim. □ 

Lemma 3. Let u^Uj :lxR->M satisfy Assumptions and Furthermore, let 

Uij(t,6) :=Ui{t,6) [ Uj(t,r)dr (B.13) 
Jo 

then there exist Mij,Lij G (0, oo) such that 

1. Uij(t,6) is T -periodic in 0, i.e. Uij(t,6 + T) = v,ij(t,9), 
2- sup t£R |uij(t,wi)| < My, 

3. \uij{tx,e) - Uij(t 2 ,e)\ <i«l*x-ta|. 

Proof. To (1): Consider Uij{t,0 + T). Performing a change of variables s = r — T and ds = dr 
yields 

Ui(t,0 + T) / Uj(t,r)dr 

(B.14) 

= Uiit, 0) I Uj(t,s + T)ds. 
J-T 

Due to Assumption AEJitj^, •) has zero average. Thus, the expression above yields 

Ui(t,0) / Uj(t,r)dr — Ui(t,0) / uj(t,r)dr. 



(B.15) 
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To (2): Since T G (0, oo) we can divide [0, 9] into n £ N Q pieces of length T such that 9 = Tn + S 
with < 5 < T being the leftover piece. Due to Assumption ^|1J the first pieces are zero. Thus, 
we obtain 



n-l Ak+1)T 
(t,9)\ = \u i (t,e)Y' / Uj (t,r)dr 

k=Q JkT 
=0 

rnT+S 

+ Ui(t,6) / Uj(t,r)dr\ 

JnT 



(B.16) 



< M i M j (9 - nT) < MiMjT, 

= :Mij 

where the last step follows from Assumption A|3] 

To (3): Using the definition of u.y in ( B.13[ ) we can add and subtract the term Ui(t\, 9) J Q Uj(t 2 , r)dr 
which yields 

\uij(tx,6) - Uij{t 2 ,0)\ 



(B.17) 



Since T £ (0, oo) we can divide [0, 9] into n £ No pieces of length T such that 9 = Tn + 6 with 
< S < T being the leftover piece. We obtain for the expression above 



K(ti,0) / (iij(tt,r) - Uj(t 2 ,r)dr) 
Jo 

+ {u l {t 1 ,9)-u l {t 2 ,9)) / Uj (t 2 ,r)dr\. 



n-l „(fc+l)T 

= \uj(t\,ff) / (v,j(ti,r) - Uj(t 2 ,r)dr) 

k=0 JkT 
j.nT+8 

+ u l (t 1 ,9) / {u s (ti,r) - Uj(t 2 ,r)dr) 

JnT 

+ (u i (t 1 ,9)-u i (t 2 ,6)) V / u 3 {t 2l r)dr 

,,._n JkT 



+ (Ui(h,9)-Ui(t 2 ,e)) 



nT+8 



Uj(t 2 ,r)dr\. 



(B.18) 



The first and third line in (B.18) sum up to zero due to Assumption A|4| Furthermore, due to 
Assumptions A|3]we obtain 



l-nT+6 

/ Ljlh-hldr 

JnT 



<\Ui(h 



nT+8 



\uj(t 2 ,r)\dr 



(B.19) 



<(Af i L i + L i Af i )tf|*i-* 2 | 

< (MiLj + LiMj)T \ti — t 2 \ 
" v ' 

— :La 
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This was the last property we had to prove. 



□ 



Lemma 4. Let Ui,Uj :lxE- 
[0, oo) such that the following inequality 



satisfy Assumptions ^[3] and Then there exist k\ , k 2 , k 3l k^ £ 



uiUi(T,0JT) / Uj(s,ujs)ds 











r 


Ui(r,6 


) / Uj(T,r)dr 


dO^jdr 


10 




Jo 





. . (t-t ) 2 ,,i-io,,l, 1 

< ki h k 2 h k 3 — + fc 4 ^r 

u; www 3 



(B.20) 



is satisfied for all to £ K and a/Z t £ [io ; oo). 

Proof. In order to use Lemmablwe add and subtract/? {t^ (r, uiT)dr = (ui (t, wt) J^ t (t, r)dr)dr 
(see (B.13)) in the norm on the left hand-side of ( |B.20| . Thus, it can be written as 



to 



u. 13 (t,ujt) - - Uy(r, 



dr + R 



(B.21) 



with 



i? := 



0JUi(r,ujT) / Uj(s,ujs)ds 
J t 

- Uij (t, ujt) I dr. 



(B.22) 



Due to Lemma [3] the expression Uij in (B.21) satisfies all assumptions needed in Lemma [2] which 
can now be applied in order to establish the existence of fci, &2 € [0, 00) such that 



1 r 

Uij (r, ujt) - — / (r, 0) dOdr 
1 Jo 

< k(t-t ) + k 2 



(B.23) 



In the following we establish an up^per bound for R. We first split up the third integration interval 
in (B.22 1, i.e. J Q Uj(r,r)dr = J Uj(r,r)dr + j uta Uj(r, r)dr and obtain 



R = 



where we introduced 



Ri 



0JUi(T, CUT) 



Uj(s, uis)ds 



to 



Uj(r, r)dr 



ut 



dr + Ru 



Ui(r, wt) / Uj(T,r)dr\dT 



(B.24) 



(B.25) 
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By the changes of variables p = ujt, dp = uidr and q — ujs, dq = uids we obtain 



R 



1 



/"'( 






- rp 


tut V 






.J UjtQ 


f U 3 


e- 


, r)dr 


J dp 









Uj(-,q)dq 



(B.26) 



Since the integration intervals with respect to r and q are now equal, we combine the two inner 
integrals and introduce I(q,p) := Uj(^, q) — q)- Furthermore, we divide [uto, ut] into n € No 
pieces of length T such that oj(t — to) = Tn + 5 with < 5 < T being the leftover piece. Thus, we 
have 



R = 



^ n-l r ujt +T(k+l) 



k=0 w *o+Tfc 



1 



uito+Tn+S r 



w Juto+Tn 



Ui{-,p) 
uj 



Ulto 

I(q,p)dq 



I{q,p)dq 
dp 



dp 



(B.27) 



For reasons which become clear later, we again split up the integration interval J^ t I(q,p)dq 

ujto+Tk 
bjto 



F 



I(q,p)dq + J% to+Tk I(q,p)dq, k = l,..., 

r ut +T(k+l) 



n and obtain R — R\ + R 2 + i?3, where we define 



Ri 



n— 1 



fc=0 JuJ t +Tk 



uto+Tk 



Ui(-,p) I(q,p)dq 



1 



ait a +Tn+S 



w Jbjt +Tn 



U)t 

U]t +Tn 



dp 



Ui(-,p) I(q,p)dq 



dp 



(B.28) 



and 



R 



3 •- 



n— l 



ujt +T(k+l) 



k=Q ut +Tk 
1 
UJ 



ujt +Tn+S 
uit +Tn 



U i(-,P) I(<l,p)dq)dp 

u Ju}t +Tk J 



U i{-,P)( I(q,p)dq)dp. 

w Juto+Tn J 



(B.29) 



Each part is now treated separately. 

For i?i we split up the second integration interval [0, u)to] by introducing I 6 No such that 
uito = Tl + e with < e < T being the left-over piece. We know from Assumption A[4]that Uj(t, •) 
has zero average. Thus (B.25) simplifies to 



Ri = 



to 



Ui (t, lit) 



Tl+e 



Tl 



Uj (t, s) ds^j dr 



(B.30) 



rTl+e 
ITI 



and with Assumption jA 3J i.e. | J^l +e Uj (r, s) ds\ < Mje the expression Uij(r, 0) :— Ui(r, 0) jj 
is bounded, Uij{r, •) is T^periodic with zero mean and 0) is Lipschitz continuous which follows 
from the same reasoning as in the proof of part (3) of Lemma [3] (i.e. ( B.17[ ) to ( B.19[) ). Thus it 
satisfies all assumptions of Lemma [2] We conclude with the first statement of Lemma |2] that there 
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exist &3, € [0,oo) such that 

\ Rl \ < ~ k ^-^) + ~ k \ (B . 31) 

UJ 

We now turn to i?2- Since uj(t, ■) is T-periodic with zero mean, we have that J^° +Tk u j{^-> q)dq z 

and therefore J^° +Tk I(q,p)dq = J^° +Tk Uj (^>?) dq k = 1, . . . ,n. The crucial point now is 
that this integral does not depend on p anymore. Thus the expression i? 2 can be written as 

n-l /-u;to+T(fc+l) r ujt +Tk 



i?2=-V/ Ui(-,p)dp Uj(-,q)dq 

U k=0 J^o+Tk w Jwto w 

^ j-ujto+Tn+8 ruto+Tn 



u Juito+Tn w Jcjto 



Ui(-,p)dp I Uj(-,q)dq. (B.32) 



Substituting r = £ , = ^ and s = ds = ^ yields 

OJ ' UJ UJ 1 UJ J 



R2 = lo / tij (r, wr) dr I Uj (s, ujs) ds 



u.i(r,ujr)dr / uj (s,ujs) ds. (B.33) 
to+iif "> to 

Since V4 is bounded by Mj € (0, 00) and both Ui,uj satisfy the conditions of Lemma [2] and 
w 0o + v - *o - T( t +1) ) = T as well as uj(t + ^ - t ) = Tk, k = 1, . . . , n, we obtain with the 
second statement of Lemma [2] that there exist k$, ke, kj € [0, 00) such that 

1 k^T kaTk , , JkyTn 



7 J _ 
2 



< fc5fc6 l'^^ + 4) + ^ r(t - t0) , (B.34) 



jwe have 

and furthermore, |«i(£, 0)| < Mi, for all t,6 GM. and all i, j = 1, . . . , m. Thus, we obtain for \R- 



where we have made use of < 5 < T and the definition of n = — — ^ — above. 

For i?3 we proceed as follows. Note that due to Assumption we have that \I(q, p) | < — \q—p\ 



-, n-l -uit +T(k+l) fp t 
\R 3 \<-J2 M * / -^\q~p\dqdp 



k=0 Ju>t +Tk Juito+Tk ^ 

Y pLuta+Tn+S rp 



CO 



Mi I -^{q-pldqdp. (B.35) 

ut +Tn Juito+Tn u 



The crucial point now is that the lower integration limits of both integrations are equal. One can 
verify that after the substitutions s = q — ujto — Tk, ds = dq and r = p — Luto — Tk, dr = dp, 
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k = 1, 



, n we obtain 



i^i < -VIE 

vfe=0 



M,;L 



5 />r 



6w 2 



T 3 n + 5 3 , 



Is — rldsdr 



where we have used < 5 < T. Involving the definition of n = — ^ — - above, we obtain 



(B.36) 



\Ra\ < 



T*MjLj(t-t ) 
6u> 



(B.37) 



With R= R 1 +R 2 + R 3 in (|B.22|), (|B.23|, (|B.31|), (|B.34|) and (|B.37|) we obtain the desired upper 

□ 



■ ■ ~ ~ ~ ~ Al • L ■ ~ ~ ~ 

bound for the left hand-side of (B.20) with k\ — k$k e , k 2 — fci + fc 3 H ^-^- + k 7 MiT, k 3 = k 2 + k 4 



and k 4 = k 5 k e T 2 . 



Appendix C. Proof of Theorem [T] 



Consider the vector field fu{t,x) — bo(t,x) 

+ J2iLibi(t,x)y/ujUi(t,ujt) in Q and note that due to Assumptions .A[T]and A[3]/ W (t, •) is con- 
tinuously differentiable and fuj(-,x) is measurable. Furthermore, with Assumption A[2] we have 
that for every compact set JC C K™ and every uj G (cjo, oo) there exist an M, L £ [0, oo) such that 
\b (t,x) + J2iLxbi(t,x)^/LJUi(t,ut)\ < M and such that \b (t,Xi) + J2iLi b i(t,Xi)y/uJUi(t,Ljt) - 
bo{t, x 2 ) - Yh=\ b i(^ x 2 )y/ujUi(t, ut)\ < \b (t, xx) - b (t, x 2 )\ + M iy /uj\ J 2™= i (bj{t,xi) - b, t (t, x 2 ))\ < 
L\x\ — 12 1) t € R,x,xi,x 2 G K. Thus, with Theorem [8] in Appendix Appendix A there exist a 
t e G (0, oo) and a unique solution x : M — > 1" of d8| that can be written as 



;(*) = x + / b (r, x) + y^ bj(r, x)\fwuj(T, lot)cIt 

Jtn ■ 1 



(C.l) 



with < G [to, to + t e ) and Xo = x(to). Since, x(t) is absolutely continuous on [to, to + t e ) we can 

>i ( T ) ; 
~dr~ 



perform a partial integration for each 6j(r, a^-y/wii^T, wt), i = 1, . , . ,m with derivative rffe '^ r ' a: '* — 



bj(T,a) ^, , dbj(T,x) 
dx dr 



and obtain 



x(t) = x + 



to 



bo(r,x) 



r-ST^l dbi(r,x) . db t {r,x) . 



i=l 



<9x 9t 
+ ^J]6 J (i,x(t))C/ l (to,i) 



(C.2) 
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with Ui(to,t) := J to Ui(r,uir)dr. Since x(t) — bo(t,x(t)) + ^^ 1 bi(t,x(t))y/ 7 0Ui(t,u>i) for almost 
all t, we obtain 



x(t) = Xq + 



where we introduced 



to 



E 



b {r,x) 
dbi(r, x) 



+ Ri + R-2 



Ri := -Vw 



dx 



bj(T, x)Uj(T,LJT)Ui(t ,T) 



dr 



E 

■i=l 
m 

E 



dbj(T,x) 
dx 

dbj(r, x) 
dr 



b (T,x) 



Ui(t ,T) 



dr 



R 2 :=^J2bi(t,x(t))Ui(t ,t). 



(C.3) 



(C.4) 
(C.5) 



Adding 



and 



subtracting 



the 



u It* E™ i Ejli+i of x)uj{r, u)T)Ui{t , r)dr yields 



with 



o(t) = Xq 



bo{r,x) 

+ w ^ [bi,bj](r,x)uj(T,uJT)Ui(to,r) 

i=l 
j=i+l 

+ Ri + i?2 + R3 + Ri 



to 



dr 



R 3 :=-u Yl 

J to i_1 



*° »=1 
t m i— 1 



dh(r,x) I dU^r) 2 
dx b ^ X >2 dr 



STSr^ db z (T,x) 

Z^Z^—a-r— b i( T > x ) 



dU l {t ,T)U ] {t 0l T) 

dr 



dr 



expression 



(C.6) 



(C.7) 



(C.8) 



and by using = Ui(T,uiT)Uj(t ,T) + Uj(T,LJT)Ui(t ,T), i = 1, . . . , m, j = 1, . . . , m. 

Note that, R3 and R4 contain the rest terms after relabeling the indices. Furthermore, R3 contains 
the terms where i = j, which is treated as a special case. 

We now turn to By assumption, the solution z : K. — > R n of ^ exists and z(t) is bounded 
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for t = \po,oo) and for all z(ta) = zq € B. Thus, z(t) that can be written as 



z(t) = Z + b Q (T,z) + ^ [bi,bj](T,z)l/ji(T)dT 



(C.9) 



j = i+l 



with t € [to, oo), ^(to) = an( l ^ji(t) as defined in ([9]). 

In the following, we show that the distance between x(t) and z(t) with z(to) = x(to) = xq 
can be made arbitrary small on a finite time interval with w chosen sufficiently large. Choose 
z(t ) — x(t Q ) = x Q G K, and since JC C B is compact and since solutions initialized in B stay 
uniformly bounded, there exists a compact set M. C R™ such that for all (q € R and all z(to) € /C 
we have z(t) e .M, t e [to,oo). Next, define a tubular set around z(t), i.e. 0(t) = {x E K" : 
\x — z{t)\ < £>}, t 6 [to,oo). Suppose now that i e < tf where [to, to + t e ) is the maximal interval 
of existence of x. Thus |x(t)| — > oo for t — > t e and by the compactness of 0(t), t € [to,oo) there 
exists a time < t e where x(t) leaves 0(t). Notice that x{i) is contained in Z/p up to time 
to + tD (see Fig. C.7). We show now that >tf for lu sufficiently large. 




o(* + *d) ; 



x(t) ; 



Figure C.7: x(t) stays in Uff for all t g [f , t + to] 

Consider the distance between x(t) and z(t) through z(to) = x(to) for t £ [to, to + to]- We add 
and subtract the expression J. [6^, 6j](r, x)vji{T)dr and obtain 



a;(t) - z(t) = / & (t, - & ( r > z) 

Jt 



i=l ^ 

+ R± + i?2 + -R3 + Ri + -R5 



(CIO) 



with 



m m „i 

R 5 :=J2 E / [^](T,*)^(T,WT)dT. 
7 = 1 -, =7 -4-1 J to 



i=l 



(C.ll) 



and Vji{T,u)r) = w%(r, uir)Ui(t ,T) - ^(r). 
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Suppose for that there exist k € [0, oo) and ujX <= (0, oo) such that for every uj € (wg, oo) we have 



|MjMjT and thus, with Assumpt 
i G (0, oo) such that \bo(r, x) — bi 

and therefore 



Ri\ < Note that with Assumption A|3 we have that |mj,*(t)| < ^ f Q \uj(T,8) J Q Ui(r,6)\ < 



)tions Afljwe have that for every compact set an there exists an 
•o(r,z)+22 m i= i ([&i,bj](r,a:)-[6i,6j](r,2:))z/ji(T)| < L\x(t)-z(t)\ 

j=i+l 



\x(t) - z(t)\ < L\x(t) - z(t)\(1t + 



(C.12) 



with x(t),z(t) G U]y , < G [^o^o + *d] and uj £ (wg,oo). Using the Lemma of Gronwall-Bcllman 
we get 



\x(t) - z(t)\ <—^e 
v uj 



L(t-t ) 



(C.13) 



Suppose now that tp < tf and choose 

uj = max{ 4 ^r f ,u '}, which yields with |Cl3| |a;(t)-z(t)| < f e Ht-t a -t,) m Thug | a ,( i )_ z ( t )| < 
I? for all t € [io,io + to] and all a; € [wg,oo) which contradicts the assumption that to is the 
time when x(t) leaves the tube 0(t). Furthermore, \x(t) — z(t)\ < D for all t € [to, to +tf] which 
contradicts tjj < tf. We conclude that for every compact set K. C B, for every D € (0, oo) and 
every G (0, oo) there exist an ujq £ (0, oo) such that for every ui £ [ojo, oo), for every to G K and 
for every ij e t there exist solutions x, z : K — ► K n through x(fo) = ^(to) = xg which satisfy 
\x(t)-z(t)\<D,te[to,to+t f }. 

It remains to show that there exist k € [0, oo) and Wq G (0, oo) such that for every uj <= (u)q, oo) 
we have Y^l=i 1^1 — Following the same lines as in [16] the expressions \Ri\,i — 1,...,5 

decay uniformly to zero with uj — > oo on compact sets. Due to space limitations, this is shown 
only for R$. The procedure is similar for R\ to i?4. 

Due to Assumption A[l] the vector fields 6,, i = 1, . . . , m are twice continuously differentiable 
and thus we can perform a partial integration which yields for R§ 



m „t 
Rh = X! foA'K*' 1 ) / Vji(T,UJT)dT 

i=1 -'to 



i=l 

i=»+i 



,K dx ' 

d[bj,bj](T,x 
8t 



V 3i {0,uj6)d8 



it, 



dr. 



(C.14) 
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Substituting ±(r) = bo(r, x(r)) +X}i=i &i( T j x ( T ))-\/ w u i( T , iL > T ) yields 



m „t 

i? 5 = [6i,&j-](t,a;) / Vji(T,ujT)dT 



8=1 

j=i+l 

'VMM 



<9x 



6 (t, x) +E 6 4 (t, at) V^Mi (r,ur) 

i=l 



Vji(6,u;6)d6 \dr. 



9[6i,bj](T,a:) 
<9t 



For every there exist due to Assumptions A[l] A[2j and A|3] 
such that l[Mj](*,s)l < d, l ^-W^ I < C 2 , \b (r,x)+ ^ bi 'f T ^ 
C4 for every r, t £ R and every ir s . This yields 

rt 



(C.15) 

constants Ci, . . . , C4 £ [0, 00) 

I < C 3 and I J]™ 1 &i(r,i)ui(r,a;T)| < 



i=l 



C2(C3 



to 









Iu0)d6 


J to 





dr. 



(C.16) 



Furthermore, the functions itj, i = 1, . . . , m satisfy the assumptions of Lemma[4]and thus, there ex- 
ist k{\ 4\ kg, fcf £ [0, 00) such that I j£ Vji(T, ujT)dr\ < k^^^ +4^-^+4^+4^ and also 

It I /to Vji{e,ue)de\dT < k[ 1 ^^- + + fcj'*^ + K 1 -^- From these estimates it 

becomes clear that for every compact set U^ 1 C R™, for every to,t,o € R there exist feo,5 € [0, 00) 
and wo,5 G (0, 00) such that for all uj £ (w ,5, 00) and for all t £ [to, t + tjj] we have |i?s| < 

Estimates for i? x and i? 2 follow immediately from Assumptions AH] to AH] and Lemma [2j For 
the expressions i?3 and R4 a partial integration yields a similar result. Thus, there exist feo,i,Wo, 
such that |i2j| < € (wo.ij 00) j i = 1, . . . , 5 respectively. Summarizing, for every compact set 

£t£> C R, for every to,t_o £ R there exist u>q = max.;{wo,i} and k — maXi{fco,i} such that for all 
lu £ (ojq, 00) and for all t £ [to, to + to] 



i = l 



< 



(C.17) 



Appendix D. Proof of Theorem [2] 

This follows the same argumentation as in [TS] but extends it to the stability of a compact set. 

Practical uniform stability We show now that S is locally practically uniformly stable for 
(l8j), see Definition 1. Take an arbitrary e £ (0, 00) and let B 2 £ (0, e). First observer that, since the 
system Q is uniformly stable with respect to S, there exists a S £ (0, 5) with lif being contained 
in the region of attraction of S for ^ such that for all to £ R 

z(t ) £Ui ^ z(t) £U§ 2 , «e[to,oo). (D.l) 

Second observe that, since the set S is uniformly attractive for (fol) and < 5 < 5 we have that for 
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every B\ G (0, 8) there exists a time tf G (0, oo) such that for all to € 

z(t ) eWf =► 2(4) GW| l5 4G [to+t/,oo), 



(D.2) 



which is true due to the fact, that < 5 < 5. Let D = min{<5 — B\,e — B2}, IC = Uf and 4/ 
determined above. Due to Theorem 1, there exists an wo,i G (0, 00) such that for all oj G (wo,i> 00 ) 
and all x(t Q ) G Uf , \x(t) - z(t)\ < D, t G [4 ,4 + t f ]. This together with ( |D.1[ ) and ( |D.2[ ) yields 
for all w G (wo,i, 00) 



a;(4 ) G Uf x{t) G , 

4 G [4 , 4 + 4/] and x(t Q + t f ) G Wf 



(D.3) 



Since a;(4o + 4/) G Uf a repeated application of the procedure with another solution z(4) of (|9| 
through z(4q + tf) yields for all to G R, for all cj G (wo.1,00) 



x(4 ) eUf ^x(t) eUf,te [t ,oo). 



(D.4) 



Practical uniform attractivity We show now that there exists a 6 G (0, 00) such that 5 is 
5-practically uniformly attractive for ([8]), see Definition 2. Choose some <5 G (0,(5) with being 
contained in the region of attraction of S for Q and choose some e G (0, 00). By practical uniform 
stability proven above, there exist C3 G (0,oo) and wo,i G (0,oo) such that for all to G K and for 
all u; G (wq.1i 00 ) 



x(4 ) G z(4) G Wf ,4 G [4 ,oo). 



(D.5) 



Let B 3 G (0,C3). Since the set S is uniformly attractive for there exists a 4^ G (0,oo) such 
that for all 4 G R 



«(t ) G Uf z(4) G WjLt G [4 + 4/, 00) 



(D.6) 



Due to Theorem 1 with K. — Uf , D = C3 — B3 and 4y defined above yields the existence of 
w o,2 G (0, 00) such that for all to G K and for all oj G (^0,2, 00) and all x(to) G Uf we have that 
\x(t) — z(4)| < D, t E [to, to + tf]. This estimate together with (D.6| yield for all to G M and for 
all oj G (cj ,2, 00) 



z(4 ) euf ^x(to + t f ) eu£ 3 . 



(D.7) 



This, together with (D.5), leads for all to G K, for all a; G (wo,2,oo) where ojo — rnaxjwo^, ^0.2} to 

x(t ) eUf ^> x(t) £Uf,t£ [t +t f , 00). (D.8) 
This is the last property we had to prove. 
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